<!DOCTYPE html>
<html lang="zh-CN">





<head>
  <meta charset="UTF-8">
  <link rel="apple-touch-icon" sizes="76x76" href="/img/favicon.png">
  <link rel="icon" type="image/png" href="/img/favicon.png">
  <meta name="viewport"
        content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no, shrink-to-fit=no">
  <meta http-equiv="x-ua-compatible" content="ie=edge">
  
  <meta name="theme-color" content="#2f4154">
  <meta name="description" content="机器学习，深度学习，强化学习，算法，Linux">
  <meta name="author" content="seven">
  <meta name="keywords" content="">
  <title>机器学习-EM - SimpleAI</title>

  <link  rel="stylesheet" href="https://cdn.staticfile.org/twitter-bootstrap/4.4.1/css/bootstrap.min.css" />
<link  rel="stylesheet" href="https://cdn.staticfile.org/github-markdown-css/4.0.0/github-markdown.min.css" />


  <link  rel="stylesheet" href="https://cdn.staticfile.org/highlight.js/10.0.0/styles/github-gist.min.css" />


<!-- 主题依赖的图标库，不要自行修改 -->

<link rel="stylesheet" href="//at.alicdn.com/t/font_1749284_fmb4a04yx8h.css">



<link rel="stylesheet" href="//at.alicdn.com/t/font_1736178_pjno9b9zyxs.css">




<link  rel="stylesheet" href="/css/main.css" />

<!-- 自定义样式保持在最底部 -->


<meta name="generator" content="Hexo 4.2.1"></head>


<body>
  <header style="height: 70vh;">
    <nav id="navbar" class="navbar fixed-top  navbar-expand-lg navbar-dark scrolling-navbar">
  <div class="container">
    <a class="navbar-brand"
       href="/">&nbsp;<strong>Simple AI</strong>&nbsp;</a>

    <button id="navbar-toggler-btn" class="navbar-toggler" type="button" data-toggle="collapse"
            data-target="#navbarSupportedContent"
            aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
      <div class="animated-icon"><span></span><span></span><span></span></div>
    </button>

    <!-- Collapsible content -->
    <div class="collapse navbar-collapse" id="navbarSupportedContent">
      <ul class="navbar-nav ml-auto text-center">
        
          
          
          
          <li class="nav-item">
            <a class="nav-link" href="/">
              <i class="iconfont icon-home-fill"></i>
              首页</a>
          </li>
        
          
          
          
          <li class="nav-item">
            <a class="nav-link" href="/archives/">
              <i class="iconfont icon-archive-fill"></i>
              归档</a>
          </li>
        
          
          
          
          <li class="nav-item">
            <a class="nav-link" href="/categories/">
              <i class="iconfont icon-category-fill"></i>
              分类</a>
          </li>
        
          
          
          
          <li class="nav-item">
            <a class="nav-link" href="/tags/">
              <i class="iconfont icon-tags-fill"></i>
              标签</a>
          </li>
        
          
          
          
          <li class="nav-item">
            <a class="nav-link" href="/about/">
              <i class="iconfont icon-user-fill"></i>
              关于</a>
          </li>
        
        
          <li class="nav-item" id="search-btn">
            <a class="nav-link" data-toggle="modal" data-target="#modalSearch">&nbsp;&nbsp;<i
                class="iconfont icon-search"></i>&nbsp;&nbsp;</a>
          </li>
        
      </ul>
    </div>
  </div>
</nav>

    <div class="view intro-2" id="background" parallax=true
         style="background: url('/img/default.png') no-repeat center center;
           background-size: cover;">
      <div class="full-bg-img">
        <div class="mask flex-center" style="background-color: rgba(0, 0, 0, 0.3)">
          <div class="container text-center white-text fadeInUp">
            <span class="h2" id="subtitle">
              
            </span>

            
              
                <div class="mt-3 post-meta">
                  <i class="iconfont icon-date-fill" aria-hidden="true"></i>
                  <time datetime="2018-07-29 10:00">
                    2018年7月29日 上午
                  </time>
                </div>
              

              <div class="mt-1">
                
                  
                  <span class="post-meta mr-2">
                    <i class="iconfont icon-chart"></i>
                    1.4k 字
                  </span>
                

                
                  
                  <span class="post-meta mr-2">
                      <i class="iconfont icon-clock-fill"></i>
                    
                    
                    16
                     分钟
                  </span>
                

                
                  <!-- 不蒜子统计文章PV -->
                  
                  <span id="busuanzi_container_page_pv" class="post-meta" style="display: none">
                    <i class="iconfont icon-eye" aria-hidden="true"></i>
                    <span id="busuanzi_value_page_pv"></span> 次
                  </span>
                
              </div>
            
          </div>

          
        </div>
      </div>
    </div>
  </header>

  <main>
    
      

<div class="container-fluid">
  <div class="row">
    <div class="d-none d-lg-block col-lg-2"></div>
    <div class="col-lg-8 nopadding-md">
      <div class="container nopadding-md" id="board-ctn">
        <div class="py-5" id="board">
          <div class="post-content mx-auto" id="post">
            
              <p class="note note-info">
                
                  本文最后更新于：2 小时前
                
              </p>
            
            <article class="markdown-body">
              <h3 id="EM算法简介"><a href="#EM算法简介" class="headerlink" title="EM算法简介"></a>EM算法简介</h3><p><strong>最大期望演算法</strong>（<strong>Expectation-maximization algorithm</strong>，又译<strong>期望最大化算法</strong>）在统计中被用于寻找，依赖于不可观察的隐性变量的概率模型中，参数的最大似然估计。</p>
<p>在<a href="https://zh.wikipedia.org/wiki/%E7%BB%9F%E8%AE%A1" target="_blank" rel="noopener">统计</a><a href="https://zh.wikipedia.org/wiki/%E8%AE%A1%E7%AE%97" target="_blank" rel="noopener">计算</a>中，<strong>最大期望（EM）算法</strong>是在<a href="https://zh.wikipedia.org/wiki/%E6%A6%82%E7%8E%87%E6%A8%A1%E5%9E%8B" target="_blank" rel="noopener">概率模型</a>中寻找<a href="https://zh.wikipedia.org/wiki/%E5%8F%82%E6%95%B0" target="_blank" rel="noopener">参数</a><a href="https://zh.wikipedia.org/wiki/%E6%9C%80%E5%A4%A7%E4%BC%BC%E7%84%B6%E4%BC%B0%E8%AE%A1" target="_blank" rel="noopener">最大似然估计</a>或者<a href="https://zh.wikipedia.org/w/index.php?title=%E6%9C%80%E5%A4%A7%E5%90%8E%E9%AA%8C%E4%BC%B0%E8%AE%A1&action=edit&redlink=1" target="_blank" rel="noopener">最大后验估计</a>的<a href="https://zh.wikipedia.org/wiki/%E7%AE%97%E6%B3%95" target="_blank" rel="noopener">算法</a>，其中概率模型依赖于无法观测的<a href="https://zh.wikipedia.org/w/index.php?title=%E9%9A%90%E6%80%A7%E5%8F%98%E9%87%8F&action=edit&redlink=1" target="_blank" rel="noopener">隐性变量</a>。最大期望算法经常用在<a href="https://zh.wikipedia.org/wiki/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0" target="_blank" rel="noopener">机器学习</a>和<a href="https://zh.wikipedia.org/wiki/%E8%AE%A1%E7%AE%97%E6%9C%BA%E8%A7%86%E8%A7%89" target="_blank" rel="noopener">计算机视觉</a>的<a href="https://zh.wikipedia.org/wiki/%E6%95%B0%E6%8D%AE%E8%81%9A%E7%B1%BB" target="_blank" rel="noopener">数据聚类</a>（Data Clustering）领域。最大期望算法经过两个步骤交替进行计算，第一步是计算期望（E），利用对隐藏变量的现有估计值，计算其最大似然估计值；第二步是最大化（M），最大化在E步上求得的<a href="https://zh.wikipedia.org/wiki/%E6%9C%80%E5%A4%A7%E4%BC%BC%E7%84%B6%E4%BC%B0%E8%AE%A1" target="_blank" rel="noopener">最大似然值</a>来计算参数的值。M步上找到的参数估计值被用于下一个E步计算中，这个过程不断交替进行。</p>
<h3 id="最大似然估计"><a href="#最大似然估计" class="headerlink" title="最大似然估计"></a>最大似然估计</h3><p><strong>注意</strong>：EM算法主要用于从不完整数据中计算最大似然估计，本身可以看成是特殊情况下计算极大似然的一种方法。 </p>
<p><strong>极大似然估计</strong>所要解决的问题是：给定一组数据和一个参数待定的模型，如何确定模型的参数。使得这个确定的参数后的模型在所有模型中产生已知数据的概率最大。（<strong>模型已定，参数未知</strong>）。</p>
<h4 id="简单举个例子："><a href="#简单举个例子：" class="headerlink" title="简单举个例子："></a><strong>简单举个例子</strong>：</h4><ul>
<li>10次抛硬币的结果是：正正反正正正反反正正</li>
<li>假设P是每次抛硬币的结果为正的概率。</li>
<li>得到这样实验结果的概率我们就可以算出来：<ul>
<li>$P=pp(1-p)ppp(1-p)(1-p)pp \\ = p^7(1-p)^3$</li>
</ul>
</li>
<li>然后很愉快的最优解就算出来了：$p=0.7$</li>
</ul>
<p>我们引申到<strong>二项分布的最大似然估计</strong></p>
<ul>
<li>在抛硬币的试验中，进行N次独立试验，n次朝上，N-n次朝下。</li>
<li>然后假定朝上的概率为p，使用对数似然函数作为目标函数：<ul>
<li>$f(n | p)=log(p^n(1-p^{N-n}))$</li>
<li>$\frac{\partial(f(n | p))}{\partial p}=\frac nP-\frac{N-n}{1-p}=0$</li>
<li>然后就可以得到概率：$p=\frac nN$</li>
</ul>
</li>
</ul>
<p>升级一下，<strong>进一步考察</strong></p>
<ul>
<li><p>若给定有一组样本$x_1,x_2,…x_n$，已知它们来自于高斯分布$N(u,\sigma)$，试估计参数$u, \sigma$.</p>
</li>
<li><p>高斯分布的概率密度函数：</p>
<ul>
<li><h4 id="f-x-frac-1-sqrt-2-pi-sigma-e-frac-x-i-u-2-2-sigma-2"><a href="#f-x-frac-1-sqrt-2-pi-sigma-e-frac-x-i-u-2-2-sigma-2" class="headerlink" title="$f(x)=\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x_i-u)^2}{2\sigma^2}}$"></a>$f(x)=\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x_i-u)^2}{2\sigma^2}}$</h4></li>
</ul>
</li>
<li><p>将$X_i$的样本值$x_i$带入，得：</p>
<ul>
<li><h4 id="L-x-prod-i-1-n-frac-1-sqrt-2-pi-sigma-e-frac-x-i-u-2-2-sigma-2"><a href="#L-x-prod-i-1-n-frac-1-sqrt-2-pi-sigma-e-frac-x-i-u-2-2-sigma-2" class="headerlink" title="$L(x)=\prod_{i=1}^n\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x_i-u)^2}{2\sigma^2}}$"></a>$L(x)=\prod_{i=1}^n\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x_i-u)^2}{2\sigma^2}}$</h4></li>
</ul>
</li>
<li><p>化简对数似然函数：</p>
<ul>
<li><h4 id="l-x-log-prod-i-1-n-frac-1-sqrt-2-pi-sigma-e-frac-x-i-u-2-2-sigma-2-sum-ilog-frac-1-sqrt-2-pi-sigma-e-frac-x-i-u-2-2-sigma-2-sum-ilog-frac-1-sqrt-2-pi-sigma-sum-i-frac-x-i-u-2-2-sigma-2-frac-n2log-2-pi-sigma-2-frac-1-2-sigma-2-sum-i-x-i-u-2"><a href="#l-x-log-prod-i-1-n-frac-1-sqrt-2-pi-sigma-e-frac-x-i-u-2-2-sigma-2-sum-ilog-frac-1-sqrt-2-pi-sigma-e-frac-x-i-u-2-2-sigma-2-sum-ilog-frac-1-sqrt-2-pi-sigma-sum-i-frac-x-i-u-2-2-sigma-2-frac-n2log-2-pi-sigma-2-frac-1-2-sigma-2-sum-i-x-i-u-2" class="headerlink" title="$l(x)=log\prod_{i=1}^n\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x_i-u)^2}{2\sigma^2}} \\ = \sum_ilog\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x_i-u)^2}{2\sigma^2}} \\ =\sum_ilog\frac{1}{\sqrt{2\pi}\sigma} + \sum_i {-\frac{(x_i-u)^2}{2\sigma^2}} \\ =-\frac n2log(2\pi\sigma^2)-\frac{1}{2\sigma^2}\sum_i(x_i-u)^2 $"></a>$l(x)=log\prod_{i=1}^n\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x_i-u)^2}{2\sigma^2}} \\ = \sum_ilog\frac{1}{\sqrt{2\pi}\sigma}e^{-\frac{(x_i-u)^2}{2\sigma^2}} \\ =\sum_ilog\frac{1}{\sqrt{2\pi}\sigma} + \sum_i {-\frac{(x_i-u)^2}{2\sigma^2}} \\ =-\frac n2log(2\pi\sigma^2)-\frac{1}{2\sigma^2}\sum_i(x_i-u)^2 $</h4></li>
</ul>
</li>
<li><p>参数估计</p>
<ul>
<li><p>目标函数：</p>
<ul>
<li><h4 id="l-x-frac-n2log-2-pi-sigma-2-frac-1-2-sigma-2-sum-i-x-i-u-2"><a href="#l-x-frac-n2log-2-pi-sigma-2-frac-1-2-sigma-2-sum-i-x-i-u-2" class="headerlink" title="$l(x)=-\frac n2log(2\pi\sigma^2)-\frac{1}{2\sigma^2}\sum_i(x_i-u)^2 $"></a>$l(x)=-\frac n2log(2\pi\sigma^2)-\frac{1}{2\sigma^2}\sum_i(x_i-u)^2 $</h4></li>
</ul>
</li>
<li><p>将目标函数对参数$u,、\sigma$分别<strong>求偏导</strong>，然后<strong>令偏导等于0</strong>，就可以得到$u,\sigma$的式子：</p>
<ul>
<li><h4 id="u-frac-1n-sum-ix-i"><a href="#u-frac-1n-sum-ix-i" class="headerlink" title="$u=\frac 1n \sum_ix_i$"></a>$u=\frac 1n \sum_ix_i$</h4></li>
<li><h4 id="sigma-2-frac-1n-sum-i-x-i-u-2"><a href="#sigma-2-frac-1n-sum-i-x-i-u-2" class="headerlink" title="$\sigma^2=\frac 1n \sum_i(x_i-u)^2$"></a>$\sigma^2=\frac 1n \sum_i(x_i-u)^2$</h4></li>
</ul>
</li>
</ul>
</li>
<li><p>到现在，我们就可以很直观的看出，这个结果是和矩估计的结果是一致的，也就是说样本的均值就是高斯分布的均值，样本的伪方差就是高斯分布的方差。</p>
</li>
</ul>
<p>这就是我们所熟知的极大似然估计。</p>
<h3 id="Jensen-不等式"><a href="#Jensen-不等式" class="headerlink" title="Jensen 不等式"></a>Jensen 不等式</h3><p>设$f$是定义域为实数的函数，如果对于所有的实数$x$。如果对于所有的实数$x$，$f(x)$的二次导数大于等于0，那么$f$是凸函数。当$x$是向量时，如果其$hessian$矩阵$H$是半正定的，那么$f$是凸函数。如果只大于0，不等于0，那么称f是严格凸函数。<br> Jensen不等式表述如下：<br> 如果$f$是凸函数，X是随机变量，那么：$E[f(X)]&gt;=f(E[X])$<br> 特别地，如果$f$是严格凸函数，当且仅当X是常量时，上式取等号。<br> 如果用图表示会很清晰：</p>
<p><img src="https://eveseven.oss-cn-shanghai.aliyuncs.com/20200530230233.png" srcset="/img/loading.gif" alt="images"></p>
<p>图中，实线$f$是凸函数，$X$是随机变量，有0.5的概率是a，有0.5的概率是b。（就像掷硬币一样）。X的期望值就是a和b的中值了，图中可以看到$E[f(X)]&gt;=f(E[X])$成立。<br> 当f是（严格）凹函数当且仅当-f是（严格）凸函数。<br> Jensen不等式应用于凹函数时，不等号方向反向。</p>
<h3 id="如何理解EM算法？"><a href="#如何理解EM算法？" class="headerlink" title="如何理解EM算法？"></a>如何理解EM算法？</h3><p>在前面我们所遇到的，一堆由已知分布得到的数据，如果模型中的变量都是可以观测到了，为了求其中的参数，这时就可以直接使用极大似然估计。</p>
<p>但是，有这么一种情况，就是当模型中含有<strong>隐变量</strong>（数据观测不到了）的时候，再使用极大似然估计的时候，计算过程就会显得极为复杂甚至不可解。当我们正一筹莫展的时候，<strong>EM算法</strong>就出现了。</p>
<h3 id="EM算法的思路"><a href="#EM算法的思路" class="headerlink" title="EM算法的思路"></a>EM算法的思路</h3><p>  <strong>输入</strong>：观测变量数据X，隐变量数据Z，联合分布$P(X,Z | \theta)$，条件分布$P(Z | X,\theta)$</p>
<p><strong>输出</strong>：模型参数$\theta$</p>
<p><img src="https://eveseven.oss-cn-shanghai.aliyuncs.com/20200530230234.png" srcset="/img/loading.gif" alt="images"></p>

            </article>
            <hr>
            <div>
              <div class="post-metas mb-3">
                
                  <div class="post-meta mr-3">
                    <i class="iconfont icon-category"></i>
                    
                      <a class="hover-with-bg" href="/categories/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0/">机器学习</a>
                    
                  </div>
                
                
                  <div class="post-meta">
                    <i class="iconfont icon-tags"></i>
                    
                      <a class="hover-with-bg" href="/tags/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0/">机器学习</a>
                    
                      <a class="hover-with-bg" href="/tags/%E6%97%A0%E7%9B%91%E7%9D%A3%E5%AD%A6%E4%B9%A0/">无监督学习</a>
                    
                      <a class="hover-with-bg" href="/tags/%E8%81%9A%E7%B1%BB/">聚类</a>
                    
                      <a class="hover-with-bg" href="/tags/EM/">EM</a>
                    
                  </div>
                
              </div>
              
                <p class="note note-warning">本博客所有文章除特别声明外，均采用 <a href="https://zh.wikipedia.org/wiki/Wikipedia:CC_BY-SA_3.0%E5%8D%8F%E8%AE%AE%E6%96%87%E6%9C%AC" target="_blank" rel="nofollow noopener noopener">CC BY-SA 3.0协议</a> 。转载请注明出处！</p>
              
              
                <div class="post-prevnext row">
                  <div class="post-prev col-6">
                    
                    
                      <a href="/2018/07/30/2018-08-01-ml-NB/">
                        <i class="iconfont icon-arrowleft"></i>
                        <span class="hidden-mobile">机器学习-朴素贝叶斯</span>
                        <span class="visible-mobile">上一篇</span>
                      </a>
                    
                  </div>
                  <div class="post-next col-6">
                    
                    
                      <a href="/2018/07/28/2018-07-29-ml-clustering/">
                        <span class="hidden-mobile">机器学习-聚类</span>
                        <span class="visible-mobile">下一篇</span>
                        <i class="iconfont icon-arrowright"></i>
                      </a>
                    
                  </div>
                </div>
              
            </div>

            
              <!-- Comments -->
              <div class="comments" id="comments">
                
                
  <script defer src="https://utteranc.es/client.js"
          repo="sevenold/blog_comments"
          issue-term="pathname"
  
          label="utterances"
    
          theme="github-dark"
          crossorigin="anonymous"
  >
  </script>


              </div>
            
          </div>
        </div>
      </div>
    </div>
    
      <div class="d-none d-lg-block col-lg-2 toc-container" id="toc-ctn">
        <div id="toc">
  <p class="toc-header"><i class="iconfont icon-list"></i>&nbsp;目录</p>
  <div id="tocbot"></div>
</div>

      </div>
    
  </div>
</div>

<!-- Custom -->


    
  </main>

  
    <a id="scroll-top-button" href="#" role="button">
      <i class="iconfont icon-arrowup" aria-hidden="true"></i>
    </a>
  

  
    <div class="modal fade" id="modalSearch" tabindex="-1" role="dialog" aria-labelledby="ModalLabel"
     aria-hidden="true">
  <div class="modal-dialog modal-dialog-scrollable modal-lg" role="document">
    <div class="modal-content">
      <div class="modal-header text-center">
        <h4 class="modal-title w-100 font-weight-bold">搜索</h4>
        <button type="button" id="local-search-close" class="close" data-dismiss="modal" aria-label="Close">
          <span aria-hidden="true">&times;</span>
        </button>
      </div>
      <div class="modal-body mx-3">
        <div class="md-form mb-5">
          <input type="text" id="local-search-input" class="form-control validate">
          <label data-error="x" data-success="v"
                 for="local-search-input">关键词</label>
        </div>
        <div class="list-group" id="local-search-result"></div>
      </div>
    </div>
  </div>
</div>
  

  

  

  <footer class="mt-5">
  <div class="text-center py-3">
    <div>
      <a href="https://ai-club.gitee.io" target="_blank" rel="nofollow noopener"><span>Seven</span></a>
      <i class="iconfont icon-love"></i>
      <a href="https://ai-club.gitee.io" target="_blank" rel="nofollow noopener"><span>算法</span></a>
    </div>
    
  <div>
    
      <!-- 不蒜子统计PV -->
      
      <span id="busuanzi_container_site_pv" style="display: none">
      总访问量 <span id="busuanzi_value_site_pv"></span> 次
    </span>
    
    
      <!-- 不蒜子统计UV -->
      
      <span id="busuanzi_container_site_uv" style="display: none">
      总访客数 <span id="busuanzi_value_site_uv"></span> 人
    </span>
    
  </div>


    

    
  </div>
</footer>

<!-- SCRIPTS -->
<script  src="https://cdn.staticfile.org/jquery/3.4.1/jquery.min.js" ></script>
<script  src="https://cdn.staticfile.org/twitter-bootstrap/4.4.1/js/bootstrap.min.js" ></script>
<script  src="/js/main.js" ></script>


  <script  src="/js/lazyload.js" ></script>



  
  <script  src="https://cdn.staticfile.org/tocbot/4.11.1/tocbot.min.js" ></script>
  <script>
    $(document).ready(function () {
      var boardCtn = $('#board-ctn');
      var boardTop = boardCtn.offset().top;

      tocbot.init({
        tocSelector: '#tocbot',
        contentSelector: '.post-content',
        headingSelector: 'h1,h2,h3,h4,h5,h6',
        linkClass: 'tocbot-link',
        activeLinkClass: 'tocbot-active-link',
        listClass: 'tocbot-list',
        isCollapsedClass: 'tocbot-is-collapsed',
        collapsibleClass: 'tocbot-is-collapsible',
        collapseDepth: 0,
        scrollSmooth: true,
        headingsOffset: -boardTop
      });
      if ($('.toc-list-item').length > 0) {
        $('#toc').css('visibility', 'visible');
      }
    });
  </script>





  <script defer src="https://cdn.staticfile.org/clipboard.js/2.0.6/clipboard.min.js" ></script>
  <script  src="/js/clipboard-use.js" ></script>



  <script defer src="https://busuanzi.ibruce.info/busuanzi/2.3/busuanzi.pure.mini.js" ></script>




<!-- Plugins -->


  
    <!-- Baidu Analytics -->
    <script defer>
      var _hmt = _hmt || [];
      (function () {
        var hm = document.createElement("script");
        hm.src = "https://hm.baidu.com/hm.js?148e1083cf73cfd11f7ffaae0cd55877";
        var s = document.getElementsByTagName("script")[0];
        s.parentNode.insertBefore(hm, s);
      })();
    </script>
  

  

  

  

  

  



  <script  src="https://cdn.staticfile.org/typed.js/2.0.11/typed.min.js" ></script>
  <script>
    var typed = new Typed('#subtitle', {
      strings: [
        '  ',
        "机器学习-EM&nbsp;",
      ],
      cursorChar: "_",
      typeSpeed: 70,
      loop: false,
    });
    typed.stop();
    $(document).ready(function () {
      $(".typed-cursor").addClass("h2");
      typed.start();
    });
  </script>



  <script  src="https://cdn.staticfile.org/anchor-js/4.2.2/anchor.min.js" ></script>
  <script>
    anchors.options = {
      placement: "right",
      visible: "hover",
      
    };
    var el = "h1,h2,h3,h4,h5,h6".split(",");
    var res = [];
    for (item of el) {
      res.push(".markdown-body > " + item)
    }
    anchors.add(res.join(", "))
  </script>



  <script  src="/js/local-search.js" ></script>
  <script>
    var path = "/local-search.xml";
    var inputArea = document.querySelector("#local-search-input");
    inputArea.onclick = function () {
      searchFunc(path, 'local-search-input', 'local-search-result');
      this.onclick = null
    }
  </script>



  <script  src="https://cdn.staticfile.org/fancybox/3.5.7/jquery.fancybox.min.js" ></script>
  <link  rel="stylesheet" href="https://cdn.staticfile.org/fancybox/3.5.7/jquery.fancybox.min.css" />

  <script>
    $('#post img:not(.no-zoom img, img[no-zoom]), img[zoom]').each(
      function () {
        var element = document.createElement('a');
        $(element).attr('data-fancybox', 'images');
        $(element).attr('href', $(this).attr('src'));
        $(this).wrap(element);
      }
    );
  </script>





  

  
    <!-- MathJax -->
    <script>
      MathJax = {
        tex: {
          inlineMath: [['$', '$'], ['\\(', '\\)']]
        },
        options: {
          renderActions: {
            findScript: [10, doc => {
              document.querySelectorAll('script[type^="math/tex"]').forEach(node => {
                const display = !!node.type.match(/; *mode=display/);
                const math = new doc.options.MathItem(node.textContent, doc.inputJax[0], display);
                const text = document.createTextNode('');
                node.parentNode.replaceChild(text, node);
                math.start = { node: text, delim: '', n: 0 };
                math.end = { node: text, delim: '', n: 0 };
                doc.math.push(math);
              });
            }, '', false],
            insertedScript: [200, () => {
              document.querySelectorAll('mjx-container').forEach(node => {
                let target = node.parentNode;
                if (target.nodeName.toLowerCase() === 'li') {
                  target.parentNode.classList.add('has-jax');
                }
              });
            }, '', false]
          }
        }
      };
    </script>

    <script async src="https://cdn.staticfile.org/mathjax/3.0.5/es5/tex-svg.js" ></script>

  














</body>
</html>
